DOCUMENT 



RESUME 



HE 000 331 



ED 031 127 

By-Astu\ Alexander W.; And Others 

Implications of a Program of Research on Student Development in Higher Education. 

American Council on Education^ Washington^ D.C. 

Pub Date 67 
Note-42p. 

EDRS Price MF-S0.25 HC-S2.20 
Descriptors -Data Collection, * Educational Research^ Higher Education, * Institutional Environment, 
♦ Measurement, ♦Performance Criteria, Student Behavior, ♦Student Development 

The 3 papers in this report cover different aspects of a longitudinal research 
program undertaken by the American Council on Education (ACE) to assess the impact 
of different college environments on student development. ACE plans to create a 
comprehensive file of data on students and institutions of higher education which will 
be updated annually. The first paper focuses on variables that will be used to 
measure institutional environments, and possible approaches to determine how these 
variables affect student performance. The second paper deals with relevant criteria 
for assessing student development, and describes student input data already 
collected from freshmen for subsequent evaluation of grade point averages in ma|or 
fields of study, overall college achievement, and performances on achievement tests. 
The hope is to evaluate changes in student values, attitudes, personality, educational 
aspirations, and vocational choice, and to relate these to college experience and to 
student behavior in society at large after graduation. The third paper discusses how 
results of research on data collected on students and their institutions may 
contribute to the matching of college-bound students and colleges. It also considers 
how to present data on a multiplicity of input, outcome, and environmental variables 
so that they may be of maximum utility in decision-making by students, counselors, and 
admission officers. (WM) 
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The American Council on Education, founded in 1918, 
is a council of educational organizations and institutions. 
Its purpose is to advance education and educational methods 
through comprehensive voluntary and cooperative action on 
the part of American educational associations, organiza- 
tions, and institutions. 
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The Council’s Office of Research was established in 
1965 to assume responsibility for conducting research on 
questions of general concern to higher education. ACE 
Research Reports are designed to expedite communication 
of the Office’s research findings to a limited number of edu- 
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Earlier versions of these papers were originally presented 
at the Dallas American Personnel and Guidance Association meetings, 
March 21, 1967 as a symposium sponsored by Commission IX of ACPA. 
These papers will appear in a forthcoming edition of Journal .of 
College Student Personnel . 



A Program of Research on Student Development 
in Higher Education^" 

Alexander W. Astin 
American Council on Education 

The American Council on Education has recently undertaken a large- 
scale program of longitudinal research on student development in higher 
education. The major objectives of this program are to assess the impact 
that different college environments have on the student* s development and 
to provide a source of current, readily available descriptive information 
about the population, of college students. It is our hope that the results 
of this research will have both theoretical and practical implications for 
college admissions, guidance, and educational administration. 

The past few years have seen a significant increase in the number 
of large-scale studies in higher education, primarily because quantities 
of data can now be collected and summarized easily and because institutions 
and students have usually been extremely cooperative. Most of these 
studies, however, have used biased or accidental samples of students and 
institutions. Many have been no more than adjuncts to ongoing operational 
programs. In both cases, these projects have tended to focus on narrow 
and specialized concerns, and thus have failed to view the student as part 
of a large, diverse, and complex higher educational system. Most studies 
of student dropouts, to cite just one example, have concentrated solely 
on the effects of personal variables, without attempting to assess the 
impact of the college environment. 



In addition, many of these project-oriented research studies are 
very costly, in that their data files are of limited usefulness in 
further research. Because of differences in measurement instruments, 
sampling techniques, and methods of subject identification, the data from 
different investigations are seldom interchangeable, and the researcher 
initiating a new project typically starts his data collection from scratch. 
Not only do such practices result in duplicative costs and make excessive 
demands on students* time, but they also mean that each new longitudinal 
study takes an unnecessarily long time to complete. 

The initial goal of the American Council on Education's research 
program is to create and maintain a comprehensive file of longitudinal 
student data from a representative sample of colleges and universities. 

In addition to student data, the files will contain comprehensive data 
concerning college environments, staff, and administrative policies. This 
file will be used in the Council’s continuing program of longitudinal 
research, and, moreover, may help other research organizations and indivi- 
dual investigators to coordinate their activities. 

The general design of the research program is shown schematically 
in Figure 1. Each box in this figure represents one of the necessary 
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Figure 1. Schematic Diagram of Studies of College Effects. 

- 2 - 



categories of information about higher educational institutions! inputs, 
outputs, and environments. Outputs are the operational manifestations of 
educational objectives. More specifically, they are those skills, atti- 
tudes, and behaviors of students that the higher educational institution 
either does influence or attempts to influence. Adequate measures of 
relevant educational outputs are, clearly, the sine qua non of meaningful 
educational research (Panos, 1967). 

Inputs are the talents, skills, aspirations, and other potentials 
for growth and learning that the student brings with him to the higher 
educational institution. These inputs are, in a sense, the raw materials 
which the institution has to work with. In collecting input information, 
the investigator must take care to measure all variables that are likely 
to affect the student’s subsequent performance on the various outputs 
under study. 

The college ’’environment” includes those aspects of the higher 
educational institution that are capable of affecting the development o £ 
the student. These include administrative policies and practices, curri- 
culum, physical plant and facilities, teaching practices, peer associations 
and other characteristics of the college environment. Although some pro- 
gress has been made in recent years, the measurement of the college environ 
ment is still in a relatively primitive state both conceptually and 
methodologically. Thus, one of the major goals of this research program 
is to improve techniques for measuring differences in college environments. 

Table 1 shows some of the measures of environmental characteristics 
that will be utilized in the research program. The first set of charac- 
teristics (listed under roman numeral I) corresponds to the standard 
administrative typologies used to classify institutions. Although these 
characteristics may be convenient to use for ordinary administrative 



Table 1 

Environmental Variables for the ACE Program of Longitudinal Research 



I. Administrative Characteristics 

A. Sex (men*s, women* s, or coed) 

B. Type (university, liberal arts college, teachers college, or 
technical institution) 

C. Control (private, private-nonsectarian, Protestant or Catholic) 

D. Geographic region 

E. Size 

F. Affluence or wealth 

G. Selectivity 

* 

II. Environmental Stimulus Factors 

A. The Peer Environment 

1. Competitiveness versus cooperativeness 

2. Arranged dating 

3. Independence 

4. Cohesiveness 

5. Informal dating 

6. Drinking versus religiousness 

7. Musical-artistic activities 

8. Femininity 

9. Leisure time 

10. Student employment 

11. Career indecision 

12. Use of automobiles 

13. Regularity of sleeping habits 

14. Use of the library 

15. Conflict with regulations 

B. The Classroom Environment 

16. Involvement in the class 

17. Verbal aggression in class 

18. Extraversion of the instructor 

19. Severity of grading 

20. Familiarity with instructor 

21. Formality of the class 

C. The Geographic Environment and Living Quarters 

22. Bigness 

23. Friendliness of the housemother 

D. The Administrative Environment 

24. Severity of administrative policy against drinking 

25. Severity of administrative policy against sex 

26. Severity of administrative policy against aggression 

27. Severity of administrative policy against cheating 

* 

III. The College Image 



1 . 


Academic competitiveness 


5. 


Snobbishness 


2. 


Concern for the individual student 


6. 


Emphasis on athletics 


3. 


School spirit 


7. 


Flexibility of the curriculum 


4. 


Permissiveness 


8. 


Emphasis on social activities 



From the Inventory of College Activities 

-4 



purposes, they probably have little bearing on student development, since 
recent research shows that the environments of institutions of even one 
given type, e.g., men's colleges, differ markedly from one another (Astin, 
1967) . 

The environmental variables listed under II were derived from a 
recent large-scale study of college environments (Astin, 1967). In this 
study, the college environment was viewed simply as a set of potential 
"stimuli." The term stimuli refers here to those events or observable 
characteristics of the college that are capable of changing the sensory 
input to the student attending the college. The environmental measures 
are thus based on the frequency of occurrence of these stimuli on the 
campus. We believe that this approach to measuring college environments 
has considerable promise, and future research will be devoted to expanding 
and refining the list of variables shown in the table. 

The third category comprises eight environmental measures (shown 
under III) relating to the college "image." These measures are based on 
items similar to those used in the CCI and CUES, although the scales were 
derived from factor analyses of item intercorrelations rather than from 
a priori scales. We have reason to believe that scales constructed in 
this manner yield a maximum amount of information from a minimum number of 
items. These college image factors seem to have only a moderate degree of 
overlap with the environmental stimulus factors; apparently, the student's 
perception of his environment is influenced by factors other than the 
environmental stimuli which confront him. 

The principal objective in our research program is to determine 

how these environmental variables affect the performance of the student, 

a relationship indicated by arrow 15 in Figure 1. From a methodological 

point of view, however, a thorough knowledge of relationships A and £ is 

-5- 



required before we can adequately interpret relationship B. It is easy to 
see, for example, that the student's output performance is determined, 
in part, by his input characteristics. To put it more simply: the 

student's talents and aspirations when he enters college play a major 
role in determining what he is able to learn and the kind of person he 
eventually becomes. But the presence of relationship A complicates the 
design. It is now well known that certain characteristics of the college 

environment are closely related to student input characteristics. There- 
fore, since the student input is likely to affect both the output and the 

college environment, it is possible for a significant relationship B to 

be mediated simply by differential student input to the various environ- 
ments. In short, any obtained relationship 15 between educational practice 
and student output is necessarily ambiguous so long as no control is 
exercised over differential student input. 

So far in our research program, we have already started to collect 
data, and we have made plans about further data collection and studies for 
the next several years. Our initial sampling unit was the institution. 

In order to select a sample, all "eligible" institutions listed by the 
U. S. Office of Education in its Education Directory were sorted into 29 
cells on the basis of known institutional characteristics that previous 
research had shown to be related to student input characteristics (Astin, 
Panos, and Creager, 1966). An institution was considered eligible if it 
was currently functioning and if it had a freshman class, or its equiva- 
lent. of at least 30 students. Under these conditions, the eligible 
population consisted of 1,968 junior colleges, colleges, and universities. 
Institutions were selected randomly within each of the 29 stratification 
cells and invited to participate in the study. 

A pilot study was conducted in the fall of 1965, when a prelimin- 
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2 

ary version of a questionnaire for collecting student input information 
was administered to each of 42,061 entering freshmen in 61 institutions. 
The full-scale study involving slightly more than a quarter of a million 
students entering a sample of 307 institutions was begun in fall, 1966. 3 
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Before the end of 1966, each participating institution received a 
complete tabulation of all data on its entering freshmen. A sample page 
from an actual institutional report is reproduced in Figure 2. The first 
three columns of data are based on the information provided by the entering 
students at this particular institution. The last three columns are the 
national norms for comparable types of institutions. These normative data 
have been differentially weighted so as to approximate the population 
parameters (Astin, Panos and Creager, 1967a). As the report shows, this 
college happens to be a men’s institution (note the column of zeros under 
"female") which enrolls students who come primarily from the middle states. 
Compared to the national university norms for male students, the educa- 
tional level of the parents of these students is very high, although their 
racial backgrounds are similar to the national norms. One of the largest 
differences between this institution and the national norms is indicated in 
the last line of data: more than twice as many students at this institu- 

tion choose "none" as their current religious preference. 

Although these reports are prepared primarily as an incentive to 
the institution to participate in the study, the national norms based on 
these data have generated considerable interest among persons concerned 
with higher education. For this reason, we prepared tables of national 
norms based on 13 different sub-categories of institutions. A sample page 
from these national norms, showing data for male students, is presented in 
Figure 3. 

In addition to input data, the research design of the program re- 
quires follow-up criterion data and data on college environments. Figure 
4 depicts our plans for collecting data for each entering class in the 
research program. The horizontal line running across the middle of the 

figure represents a time dimension beginning on the left with matriculation 
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Figure 4. Schedule of Data Collection for the ACE Program of Longitudinal Research. 



and ending at some indefinite time in the future after graduation. The 
boxes formed by solid lines represent different stages in the data collec- 
tion process. The large box on the left, for example, shows the annual 
survey of entering freshmen which produces the data shown in the two 
previous figures. Although the primary purpose of this survey is to 
collect input data for the longitudinal studies, it also produces a kind 
of informational "fall-out" in the form of national norms which will be 
published each fall. After a succession of such surveys, it will be 
possible to plot trends in the characteristics of students entering 
different kinds of institutions. 

The three smaller boxes to the right of the annual survey of 
entering freshmen represent follow-ups that will be performed by the 
participating institutions. Each fall the institution will receive a 
list of its students who would normally be expected to return for another 
year. Students who did return will be checked off, and the list returned 
to the Council. These annual follow-ups will yield basic data for studies 
on dropouts. The more detailed criterion data on the students* develop- 
ment will be collected at the time of the four-year graduation follow-up. 
In addition, this follow-up will make available data to compute national 
norms for graduating seniors; it can also be used as input data for 
longitudinal studies of the impact of graduate and professional education. 
For two-year institutions, the two-year follow-up at the end of the sopho- 
more year will serve the same function as the four-year graduation follow- 
up for students entering four-year institutions. 

The purpose of the ad hoc follow-ups (shown below the line) will 
be to collect information on the college environment and also on special 
subsamples of students that have been singled out for detailed study. One 

such study in progress involves all of the twins in the 1966 survey of 
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entering freshmen. The number and extent of these special follow-ups will, 
of course, depend upon financial resources and staff interests. 

One of the objectives of the research program is to make the data 
files available to other researchers who wish to perform analyses that 
differ from those performed or planned by the ACE staff. If outside users 
are to have ready access to the data files, however, we must prepare a 
library of flexible programs to perform many of the standard types of data 
manipulations that are likely to be requested. This "software package" 
will include routines for computing summary statistics, cross-tabulations, 
multivariate analyses, and so forth. It is our intention to automate out- 
side requests for special analyses by developing a file system and a 
software package that is thoroughly documented for use by others. Although 
such a system means that special requests must be formulated so as to fit 
the available file and program library, it has the advantage of permitting 
easy and rapid access to the files and of requiring the user to define 
his needs in very explicit terms. It it our hope that such an accessing 
system will be available within the next year. 

The file has many other potential uses which are currently being 
explored. It may be possible, for instance, to link up the ACE data with 
data collected by other organizations involved in large-scale studies of 
students--national testing organizations, perhaps, whose data on high 
school students might, in combination with our data on entering freshmen, 
throw light on the process of college choice. The contribution that such 
a data file--and the studies growing out of it — could make to higher 
education are impressive. It is to be hoped that the design of our 
research program and the studies performed or planned so far constitute a 
firm forward step in realizing these possibilities. 
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Footnotes 



The first of three papers presented as a symposium at the 1967 meetings 
of the American Personnel and Guidance Association. 

2 

We are indebted to the Executive Committee of the American Association 
of Collegiate Registrars and Admissions Officers for their assistance in 
developing this questionnaire. 

3 

The 1967 survey includes approximately 300,000 students at 366 insti- 
tutions, including 97% of the institutions that participated in the 1966 
survey. 
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Criteria of Student Development 
Robert J. Panos 

American Council on Education 

Any research program which involves the outcomes of higher educa- 
tion must assess student behaviors that are relevant to the goals of the 
educational enterprise and relate these behaviors to the dynamic influences 
that characterize the college environment. Because of the extensive (and 
expensive) longitudinal research required in such an undertaking, few 
studies of a comprehensive and representative nature have been carried out. 
Moreover, a number of difficult problems in the general area remain largely 
unsolved. These larger problems can be subsumed under three broad--but 
clearly not independent--categories . First, there is the problem of defi- 
nition: What are the relevant criteria of student development? Second, 

there is the question of measurement: What observations of student be- 

havior, in what social context, best elicit the relevant data? Third, 
there is the problem of research design: What methodology of inferential 

procedures will organize and display the interrelations among the original 
observations in the most intellectually satisfying and objectively con- 
vincing manner? 

The more difficult problems of measurement and research design--at 
least as they bear on the Council's research program--have been discussed 
elsewhere (Astin, Panos, and Creager, 1966). The problem of definition 
is somewhat less difficult because, being primarily analytical, it is 
more speculative than empirical. For this reason, my major focus will be 
correspondingly hypothetical. 

Asking which criteria are relevant in assessing student develop- 
ment is equivalent to asking what the objectives of higher education are. 



More specifically we may ask: What particular student behaviors are the 

various educational interventions intended to bring about? However, equating 
"relevant criteria" with "objectives of higher education" by no means sim- 
plifies the matter. There are at least two approaches one can utilize in 
attempting to come to grips with this definitional problem. One way is to 
specify, on the basis of the statement of an educational objective, the 
operational (that is, the behavioral) manifestations of the outcome under 
consideration and the social context in which that behavior is supposed 
to occur. The verbal specification of the objective is the conceptual 

criterion (Astin, 1964). A criterion performance , then, can be conceived 
as any transactional event between an individual and his environment that 
is judged to be relevant to the conceptual criterion. Observations of the 
criterion performance, of themselves or after statistical manipulation, 
become criterion measures . This approach, of course, is the traditional 
process used in evaluation. 

Educational objectives, however, because they are proposed by di- 
verse groups--for example, administrators, teachers, students, and subject- 
matter specialists --and because they develop gradually, are initially 
poorly defined and sometimes even contradictory. Unfortunately, they 
usually remain so. The ideal method for ascertaining the relevant cri- 
teria of an educational intervention would be to deduce them from state- 
ments of educational objectives. But, unfortunately these statements--as 
they are usually articulated at the level of higher education--are too 
global and too abstract to function as a source of viable conceptual cri- 
teria. 

The fact is that educational goals, as stated in college catalogs 
for example, consist largely of superficial and essentially nonfunctional 

utterances about educating students to value the "intellectual" life, and 
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the like. Although most persons would certainly agree that such goals 
are relevant and desirable, it is obvious that the terms in such state- 
ments mean different things to different people. The task of translating 
such nonfunctional conceptual criteria into specifiable operations is 
very nearly impossible. Effort in this area is better devoted to obtaining 
a clear picture of what actually happens to the students. Perhaps, after 
we have been able to discover and adequately document what the outcomes of 
college are, we can think about whether we like them or not and what we 
can or cannot do about them. 

This brings us to a second approach to specifying the outcomes of 
higher education. In this approach, the global content of abstract state- 
ments about educational goals are arrayed under similarly global and ab- 
stract labels into areas of research interest. By utilizing such a clas- 

Behavioral Domain: 



Source of 
Criterion Measure : 
Hypothetical 
Construct 



Observable 

Behavior 



Figure 1. Scheme for classifying types of criteria of student 
development . 
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sification scheme, we can then specify criterion performances within the 



context of a particular study or research program. Figure 1 represents 
the results of one such attempt to sort our relevant criteria of student 
development into a simple 2x2 classification. 

In Figure 1, the so-called behavioral domain has been divided into 
the traditional categories of intellective, or cognitive, outcomes and 
noninte 1 lec tive , or affective, outcomes. The cognitive domain includes 
such outcomes as the student's knowledge, abilities, and intelligence, 
while the affective domain includes his motivation, values and attitudes. 

Criterion measures can be divided into two groups: those obser- 

vations which, in themselves, take the form of measures; and those obser- 
vations that imply a measure of some hypothetical latent (psychological) 
construct. Measures of the first kind include such empirically verifiable 
or observable outcomes as whether or not the student obtained a terminal 
degree and at what level, his extracurricular achievements and awards, and 
some aspects of his overt behavior toward his fellow man. With the second 
kind of measure, the outcome can only be inferred, not directly observed, 
and a person's position on a hypothetical latent continuum can only be es- 
timated. Measures of the second kind include such personality "traits" 
as intelligence, values, and attitudes. The crucial distinction operating 
in this classification is between directly observable events and outcomes 
which are not completely revealed by any set of observable indicators. 

Although this second approach apparently ignores important poten- 
tial interactions--f or example, between the cognitive and affective do- 
mains, or between a person's aptitude, values, and interpersonal rela- 
tions and his actual achievements --the classification is intended not as 
a representation of reality, but rather as a heuristic tool. Its function 
is to force us to acknowledge explicitly those criteria that we have in- 
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eluded in our research design and--perhaps more importantly--to recognize 
potentially relevant behavioral outcomes that have been excluded by the 
design, or that at least have not been explicitly dealt with in a partic- 
ular study or research program. 

It is essential to note that regardless of their source, conceptual 
criteria (educational objectives) are not subject to empirical verification 
or tests. They are rational statements of desired behavioral or social 
outcomes. In short, they represent normative assuptions about the nature 
of man, the nature of knowledge, and the nature of reality. Thus, educa- 
tional criteria can be accepted or rejected only on rational grounds. 
Similarly, the relevance or irrelevance of the designated criterion per- 
formance is a matter of judgment; that is, relevance is not empirically 
testable. 

The point, of course, is that the consumer or potential user of 
the findings of any research that has implications for educational prac- 
tice has a special responsibility not only to scrutinize the research 
design and methodology that produced the findings, but also to determine 
the relevance of the educational outcome, as defined in the research, to 
his immediate concerns. In other words, the criterion definition 
should make sense with regard both to the problem being investigated and 
to the possible applications of the findings. 

Because, of the time needed to conduct most educational research, 
it is important that research projects be designed to incorporate those 
educational objectives that are relevant to the ongoing educational pro- 
cess and that the research program be sufficiently flexible so that edu- 
cational outcomes not considered in the original design can be included 
at subsequent stages. For these reasons, all persons interested in the 
study of higher education should communicate and collaborate with one 
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another j this is crucial during the early planning stages, and clearly 
necessary throughout the conduct of the research. 

Obviously, the problems of which student outcomes are relevant to 
the goals of higher education and of how they can best be evaluated are 
far from being solved. And just as obviously, general and abstract dis- 
cussions (such as this one) will not solve them. It might be useful, 
however, to indicate briefly the types of items that we at the American 
Council on Education have included in our freshmen input surveys. These 
items are designed to provide a frame of reference within which the sub- 
sequent behavior of the student can be related. 

We have collected student input data (see box A of Figure 1) rele- 
vant to assessing such outcomes as over-all college grade point average, 
grade point average in major field of study, and subsequent performances 
on achievement tests like the area tests of the Graduate Record Examina- 
tion. Although the prediction of these kinds of intellective outcomes 
has a history of more than fifty years of research, our success at studying 
this matter remains at a level far below that which is theoretically at- 
tainable (Bloom and Peters, 1961). 

In addition, we hope to evaluate changes in the areas of student 
values, attitudes, personality, and educational aspirations (see box B) . 
Input information concerning the student's self-concept was collected by 
means of a trait self-rating technique, and information concerning his 
value-orientation was obtained from a rating scale of life goals. These 
instruments include items that can be conceptually referred to both inter- 
personal and noninterpersonal behaviors. By repeating these items in 
follow-up studies, we will be able to evaluate how the student changes 
over time and to relate these observations to the college experience. In 

the area of attitudes, the freshman survey contains input items that will 
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permit us to study how the college environment shapes the student's per- 
ception of and attitude toward his college. 

We also hope to shed some light on the process of vocational 
choice (see box C) . For example, Astin (1965) --basing his hypothesis on 
a theory of selective environmental reinforcement--has suggested that the 
student's development during college may be largely determined by the 
types of peer relationships available to him on the campus. Thus, the 
student's career choice tends to shift in the direction of the dominant 
or modal choice of his fellow students. We also hope to evaluate such 
outcomes as relatively high-level extracurricular achievements (for example, 
awards in the arts and sciences), level of final degree attained, and the 
personal and environmental factors associated with persistence in college. 

Box D presents the most difficult problem because it is here that 
outcomes of a lasting nature--relatively permanent changes in the student's 
behavior vis-li-vis society at large--are arrayed. These types of outcomes 
are usually not manifest in the student's behavior until well after he 
has left college. In the freshman survey, we have included a large assort- 
ment of input items that will permit a controlled evaluation of what may 
be called day-to-day interpersonal and noninterpersonal behaviors. Al- 
though some of these items--for instance, those relating to gambling, 
drinking beer, participating in organized demonstrations, smoking cigarettes, 
or, for that matter, praying- -may not represent kinds of behavior that a 
college curriculum explicitly attempts to influence, they are, nevertheless, 
possible outcomes of the college experience. In short, in all of our 
studies, we should not forget that educational interventions have a variety 
of consequences, and that unplanned -for outcomes --including these "side 
ef fects"--are necessarily part of the educational experience. 

In our research, we begin with the assumption that there is no 
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one method for effecting change in student behavior. We are primarily 
concerned with examining how changes in student behavior come about. 

Thus, our focus is on the process of the student’s development and on 
the identification and isolation of critical variables. We believe that 
extensive longitudinal research of a comprehensive nature is necessary 
in order to make finer distinctions among students and their college 
experiences- -distinctions which should be of use to institutions, guidance 
personnel, a"<l instructors in mapping out a strategy of learning for 
their particular students. In short, we believe that a thorough know- 
ledge of the student, his environment for learning, and his development 
during the college years will help us to discover which educational ob- 
jectives--implicit or explicit--are actually being achieved. Only then 
can we go on to decide whether these are the objectives that should be 
pursued and, if not, what can or cannot be done about it. 
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Use of Research Results in Matching Students and Colleges 

John A. Creager 
American Council on Education 

The American Council on Education's program of longitudinal research 
on the higher educational system provides special opportunities for study- 
ing the characteristics of students and of the institutions they attend. 
These opportunities are based upon the wide range of data obtained from 
a nationwide representative sample (Astin, Panos, Creager, 1966; 1967). 

The amount of such data provides the further opportunity to study inter- 
actions between the students and their college environments. One goal of 
these studies is to provide objective, factual information that will be 
relevant and helpful in solving the problem of matching the college-bound 
student with an institution of higher education. This paper is concerned 
with how such information can best be evaluated, summarized, and presented 
so that students, counselors, and admissions officers can make rational 
use of research results. 

Let us assume that the principal objective of matching students with 
colleges is to maximize output criteria or educational objectives such as 
persistence in college, motivation for graduate study, realistic career 
choice, high academic performance, mental health, knowledge and acquired 
skills, and certain values and interests. Figure 1 shows that such out- 
puts are a function of three possible factors: the main effect of student 

input, the main effect of college environment, and the interaction effects 
from a particular match between student and institution. 
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Elaboration of ACE Longitudinal Research Design 



Suppose we have as an input variable some measure of the student's 
ability and past achievement, e.g., his College Entrance Examination 
Board scores. Suppose we also have a measure of a particular college's 
academic level, e.g., the average ability of the students attending 
that institution. We wish to estimate the probability that a student with 
a certain set of C.E.E.B. scores who attends an institution with a certain 
academic level will complete his baccalaureate degree. By noting what 
happens to many students attending many different institutions, we can 
calculate a weighted sum of the freshman ability score and of the academic 
level of the insticution that will maximize prediction of graduation. 

Such a prediction is represented by the simple model of main effects shown 
in equation 1: 



Baccalaureate 

Completion 



Ability 



Intellectual ism 




(Predicted 

Outcome) 



W 1 X SI 



W 2 X CE 



(Student Input 
Main Effect) 



(College Environment 
Main Effect) 



The first term takes into account one of the main effects: that the 

brighter the student, the more likely he is to graduate. Students in 
institutions of higher academic level may or may not be more likely to 
graduate; in either case, this main effect of the college is taken into 
account by the second term of the equation. Of course, the brighter 
students are more likely to attend institutions of a higher academic 
level. This fact, which is reflected in the correlation between the 
two predictors, is taken into account when the weights are computed for 
the prediction equation. 

Beyond this, however, there are additional effects which we have 

not yet considered. For example, the effect of an interaction between 
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the student and his college environment must be considered in the context 
of prediction. We are concerned with the effects of particular matches 
between the student and his environment. To return to our example, a 
bright student matched with an institution of high academic level may- 
find this environment highly stimulating. He also contributes to the 
environment for other students attending that institution. Conversely, 
a dull student may find such an institution discouraging, or a bright 
student may become disgusted if he attends an institution that fails to 
provide an intellectually stimulating environment. The weights in the 
first prediction equation may not be the same for these different student- 
college matches. Conceivably, and for each outcome, we could arrive at 
different prediction equations for every possible combination of pre- 
dictors, but this degree of precision would probably be impractical. 
Fortunately, the problem can be handled by introducing one more term 
into our prediction equation for these matches, as shown in equation 2. 
This term is the product of the two scores and will receive its appro- 
priate weight in maximizing the prediction of the outcome, completing 
the baccalaureate degree: 



Baccalaureate 

Completion 



Ability 



Intellectual ism 



( 2 ) 



II 



W 3 X SI 



W 4 X CE 



W 5 X S1 X CE 



(Predicted 

Outcome) 



(Student Input 
Main Effect) 



(College (Interaction 

Environment between Student 
Main Effect) Input and College 

Environment) 



The American Council on Education's research program is concerned 

not only with developing equations that maximize prediction of practical 

outcomes, but also with fostering a better understanding of the higher 

education system. One way to do this is to compare the efficiency of the 
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difference predictions that result from alternately including and exclud- 
ing various terms of the prediction equation (Bottenberg and Ward, 1963). 
Suppose we found that omitting the interaction term leads to little or 
no loss of prediction. We could then conclude that the various kinds 
of interactions mentioned earlier are not relevant to predicting this 
particular outcome, in which case we could return to the simpler 
equation #1 for prediction. Suppose, however, omission of the interaction 
term did make a substantial difference. Not only would we want to keep 
the term in the prediction, but also we would then be interested in 
what different types of matches make the difference. This question could 
be investigated either by sorting the data into the different interaction 
types or by generating and investigating a more elaborate prediction 
equation involving the interaction between levels of each types of predictor. 

We have been dealing with one characteristic of the student (his 
college entrance composite), one institutional characteristic (the average 
academic ability of the student body), and one outcome (baccalaureate 
completion) . Neither the student nor the guidance counselor nor the 
admissions officer should be so naive as to suppose that these three 
variables are sufficient to make a final decision about what college 
the student should attend. The student has many characteristics; he has 
patterns of abilities, interests, aspirations, and financial resources 
which are very similar to those of many other students. The institution 
has many characteristics that may affect student development; it is one 
of several institutions with a similar pattern of student input charac- 
teristics, administrative policies, and physical and academic facilities. 

The multiplicity of characteristics to be taken into account implies the 
need for prediction equations of a more complex nature, such as equation 3. 
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ability achieve- 
ment 



aspiration intellec- size policy 

tualism 



(3) Y = (W 1 X 1 + W 2 Y 2 + 




Student Input Profile 



College Environment Profile 





Student ability 
by intellectual 
ism of student 
body 



by size of 
college 



Student 

ability 



Student high school 
achievement by size 
of college 



The many different possible outcomes should be considered differ- 
ently, according to the needs of the user; e.g., the student aspiring 
to a career that does not require graduate education will not need to 
give as much weight to prediction of that criterion as will the student 
who wants to go into scientific research or certain other professional 
careers. How, then, are we to select, digest, and present the vast 
information involving so many input, environmental, and outcome variables? 
Some of the ACE research studies are aimed to meet this need. By studying 
the redundancy of information, we can reduce the number of variables to 
those which make independent contributions to the prediction of relevant 
outcomes. We can ascertain which student environment interactions are 
important in prediction and which are not. We can determine which pro- 
files or combinations of student and institutional characteristics yield 
the same prediction and can therefore be treated the same way in making 
personnel decisions. 

To demonstrate how such information might be presented in a maxi- 
mally useful way, let me relate a dream. As with most dreams, there is 
no way to be sure that the contents will come true in the form they were 
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dreamt. Unlike most dreams, which are alleged to represent riotous 
symbolism f::cm unconscious processes, this one has both coherence and 
practical possibilities. 

In the dream a few years of carefully done and well- supported research 
produced a document about the size of the Chicago telephone directory. 

This directory is divided into two sections. The first, or main, section 
is designed to be used by the high school guidance counselor during his 
discussions with a student. The second section, the "yellow pages," are 
designed to be used by the college admissions officer. At the very back 
are comprehensive indexes to student and college environment profiles. 

Let us look first at the main section which contains a page, or a group 
of pages, giving information on each type of student. In the first scene 
of the dream, the counselor is interviewing a student concerning whom 
he has obtained information from school records, test scores, and prior 
interviews . 

A typical page of the main section of the directory is shown 
schematically in Figure 2 for a white male Protestant with college board 
scores of 600. The student being interviewed has aspirations to become 
a medical doctor, but he is not sure whether he has the financial 
resources and the persistence to fulfill his aspirations. 

He realizes he should consider being a medical technician. Even if he 
were sure he could complete graduate study, he is still not certain that 
he wants to go into clinical practice: perhaps he should become a 

research biologist. ‘Such possible outcomes are listed on the left, 
designating rows of the table. During the session, the counselor and 
the student discuss a number of institutions, all of which are geographi- 
cally and financially feasible for this student, but which differ from one 

another in the environments they provide and in the outcome probabilities 
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Subsection for Student Profile No. 117 



Student Characteristics: 




that they imply for a student with this set of characterisitcs . These 
institutions may be looked up in the index to determine their environ- 
mental profiles. For example, college A may be a large public university 
with liberal administrative policies, broad curricular offerings, and a 

student body which has a high intellectual level, but is given to consider 
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able Joe-College carousing. The index says this profile is number 5, so we 
look in column 5 to find what outcome is predicted if this student were to 
to to college A. College B may be a small, Protestant, liberal arts 
college with some scientific curricula, severe administrative policies 
against drinking, a student body of moderately high intellectual level, 
and an unusually excellent and well-used library. The college environmental 
profile index says this is profile number 37, so we look in column 37 for 
the output information that would be predicted if the student were to go 
to this college. 

One might wonder why this table does not have a column for every 
college or a row for every student. The obvious reason is that the direc- 
tory would become too big. Thus, one phase of our research involves a 
taxonomy of colleges in terms of their environmental characteristics, and 
a taxonomy of students in terms of their personal characteristics. But 
there is another and more important practical reason for this classifica- 
tion. Because they are uncertain that their applications will be accepted 
by a particular college, many students hedge their bets and submit multiple 
applications. By knowing which other colleges have similar profiles and, 
hence, a similar set of predicted outcomes, some rationality may be intro- 
duced into this shotgun approach. Such information can be combined with 
other considerations, e.g., financial and geographical, in deciding to 
which colleges to apply. 

Going down the columns 5 and 37 for the profiles of colleges A and 

B, we find that the probability that this kind of student will complete 

a baccalaureate degree is .4 at college A and .7 at college B. However, 

the counselor does not tell the student that college B, or any of the 

colleges having profile 37, is necessarily "better" than college A, or 

any of the colleges in profile 5. He notes that graduates of college A 
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are more likely to enter graduate study (.3 versus .2), and that a 
greater proportion of students like him at college A still want to be 
physicians upon graduation. If the student were absolutely sure that 
he was bound for medical school, and if the choice were really his, 
college A would seem to be the better bet, provided he were willing to 
work hard . 

But remember that this student* s career goals are not absolutely 
crystallized and that he recognizes that the need for some flexibility in 
his planning. He may want to look at the rows for other career choices 
such as research biologist and medical technician. The counselor might 
then tell the student, "college A is probably better for one of your goals 
and college B for another. Have you consir 1 iied college C (profile number 
23)? Although it ife not quite as appropriate for any one of your goals, 
it does offer favorable chances on several." In other words, the directory 
would not make final, automated personnel decisions, nor should it. What 
it does provide is objective information in readily available form to aid 
the counselor in his discussions with the student. 

How does such a table relate to the prediction equations involving 
main effects and interactions? The right hand side of the table shows the 
outcomes predicted from the student input main effect regardless of what 
college this student attends. For a given outcome, however, the values in 
the rows will vary considerably wherever the main effects of the college 
environment or the effects of the student-by-college interactions add 

to the predictability of the outcome. This is more likely to occur with 
students and institutions having jagged profiles, emphasizing special 
characteristics of either. One further point needs to be made about the 
values tabulated. Where the outcomes are categorical, the tabled value'’ 
are predicted probabilities of membership in a category, as we have been 
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assuming. If the outcome is a continuous variable, such as scores on the 
Graduate Record Examination, the tabled values may either by predicted 
scores on the outcome variable, or probabilities that the student's score 
will be above a certain level. 

Now let us move on to the next scene in the dream. An admissions 
officer at a private sectarian college has more applicants than dormitory 
space. In its educational objectives, this institution is less concerned 
with turning out students who will go on to graduate school than it is 
with developing certain religious and moral values in its students. 
Moreover, because of its faculty and physical facilities, it is better 
equipped to educate some career groups than others. At any rate, the 
admissions officer is forced to be selective. What does he do? He "lets 
the Yellow Pages do the walking." 

Figure 3 shows the schematic layout for a typical yellow page of the 
directory. Since each page in this section refers to a different environ- 
mental profile, the admissions officer chooses the page that bests describes 
his own institution. Each column now represents some outcome that this 
institution might consider representative of its educational capabilities 
and goals. The admissions officer goes down these columns to find predicted 
outcomes in favorable ranges; he then looks over to the left hand column 
in order to find out which student profiles to prefer. Having read the 
profile numbers, he can, if he wishes, refer to the student profile index 
and there obtain a description of the typical student who has such a profile. 

Like the guidance counselor, the admissions officer must guard 

against blindly using any single index to make a final personnel decision. 

The directory information is partial input to be weighed into whatever 

selection process he is using. Excessive concern about achieving one 

admissions goal may be inconsistent with achieving another goal, perhaps 
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Subsection for College Environment Profile No. 23 
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equally important. He would thus be well advised to look at more than 
one outcome column of the yellow page applicable to his institution. 

This kind of table relates to the prediction equations involving 
main effects and interactions in much the same way as do the tables in 
the main section. There is one difference. Here, a row at the very 
bottom of the table presents predicted outcomes on the various criteria 
for this type of institution, no matter what student input profile is 
considered. This row shows the outcomes predicted only from the main 
effect of that particular college environment profile. For a given 
outcome, values in the columns will vary considerably where the student 
profile main effects and the effects of student-by-college interactions 
add to the predicability of that outcome. 

Such a directory has considerable potential for dealing with still 
other kinds of problems. For example, suppose the student has already 
decided on an institution (the father is an illustrious alumnus and 
junior is going to alma mater no matter what), and the counseling interview 
is focused on choice of major field or later career. In this situation, 
either section of the directory can be consulted to ascertain those field 
and career choices that seem most promising, given this combination of 
student and college profile. 

In another example an admissions officer may have a fixed pool 

of candidates for admission: the range of student profiles is limited, 

perhaps, by certian geographic and economic factors, or by current 

admissions policy. Perhaps members of the administration and faculty are 

giving serious thought to the capabilities and educational goals of the 

institution. Should a certain department be added or expanded? Should 

certain changes be made to bring administrative policy more in line with 

the types of students who normally enter this institution? If certain 
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changes are made, which goals will be more nearly reached and which less 
so? Directory information would provide objective information relevant 
to such decisions. 

The guidance counselor working with undergraduates would also find 
such a directory useful, for example, in assisting students who are 
contemplating transfer to another institution. The directory would also 
help him in counseling students considering possible changes either in 
major field or in choice of career, in which case the predicted implica- 
tions of the various changes would be relevant. 

On awakening from such a dream, one may have doubts as to whether he 
would like to see the dream become a reality. It remains to consider 
some of these doubts. 

The tabled values in the directory do not represent a fixed set of 
expected outcomes given a certain match between student and college; 
rather, they are only estimates of such outcomes. Less than perfect 
estimates may result from unreliable measurement of student character- 
istics, of college characteristics, or of outcomes. Imperfect estimates 
may also result from failure to exhaust the relevant student input and 
college charactersitics and their possible interactions. Should we 
not have some indication of how good the estimates are so that we can 
take this into account in making personnel decisions? Yes: the size of 

the multiple correlation (or its square, the coefficient of determination) 
is obtained during the research operations that produce each predicted 
outcome. This statistic could be shown parenthetically after each esti- 
mated outcome, or otherwise tabulated in some prominent place. Reports 
of the supporting research should also document such information about 
the trustworthiness of the results. 

Would counselors and admissions officers misuse or ignore such a 
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directory? Potential users, being human, will differ in their insight 
into the scope and limitations of the directory. But then, one can 
misuse the telephone directory, or ignore it, obtaining information from 
other sources, and then proceed to misdial. In a free society, one 
is free to err. One is also free to learn with experience and to correct 
mistakes - surely more rational alternatives than disregarding informa- 
tion or burning directories. Counselors and admissions officers are 
professional people and can reasonably be expected to welcome a pro- 
fessional tool and to use it insightfully. 

VI i 1 1 extensive use of the directory, over time, result in more 
homogeneous college environments with more homogeneous student bodies 
and more homogeneous pools of alumni? Rigid and uniform nationwide use 
of the directory to the exclusion of all other factors would indeed carry 
such a risk. More flexible and rational use of the directory, as well 
as differential degree of use by counselors and admissions officers 
in various places seems a more realistic expectation. Counter-measures, 
both rational and otherwise, may be expected to arise, if indeed such 
greater homogeneity is encountered and judged to be operationally 
undesirable . 

Characteristics of students and colleges change over time. Won't 
the directory become obsolete? Yes, it must be updated. This necessity 
implies continued research to establish information about normative trends 
and to monitor the regression equations for prediction of outcomes. 

Will the directory take all of the relevant outcomes into considera- 
tion? No, of course not. In principle, any extensive demand for the 
prediction of some outcome can be met, provided that the outcome is 
sufficeintly well defined so that its occurrence or nonoccurrence can 
be determined, ana that it is predictable from student and college 
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characteristics. The outcome does not have to be a favorable one, or 
even one that is intended by either students or by educators. It does 
have to be operationally manifest in some sense, although not necessarily 
"measurable." It cannot be some vague, global verbalism. 

Doesn't the directory make invidious comparisons among institutions? 
Part of the utility of the directory is that it permits us to compare 
institutions with respect to different outcomes, separately for different 
types of students who attend them. In what sense is such comparison 
invidious? Comparison of institutions is inherent in any problem of 
college choice. Why not base such comparisons on objective information 
rather than on hearsay or folklore? If it has been shown that a certain 
institution usually fails to produce a certain desired outcome on a 
certain type of student, how is this invidious? If the institution 
does not claim to provide that particular service, it cannot be faulted; 
if it does make such a claim, it is guilty of public irresponsibility 
and can correct the matter either by relinquishing the false claim or by 
making the necessary institutional changes to validate the claim. 

For some groups, of institutions, certain student types and certain 
outcomes may be irrelevant. An example would be the choice of nursing as 
a career by students in a men's college. Even though an occasional 
graduate may in fact choose to be a male nurse, a properly designed 
directory would not present probability estimates for such an unlikely 
combination of student, college, and outcome types. 

Whether such a dream content becomes manifest as a part of higher 

education in the convenient form described here, such obviously useful 

information needs to be disseminated in some well-organized and easily 

usable fashion. To provide this information, we must have a sound base 

of longitudinal research in higher education like that being conducted 
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by the American Council on Education. The results of such a program 
can be used to aid decision making by counselors, students, admission 
officers, and other concerned v?ith the general problem of matching 
students and colleges. 
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